improvement(files): update execution for passing base64 strings by icecrasher321 · Pull Request #2906 · simstudioai/sim

icecrasher321 · 2026-01-20T21:59:26Z

Summary

Hydrate base64 strings into outputs during execution.

Type of Change

New feature

Testing

Tested manually

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-01-20T21:59:31Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
docs	Ready	Preview, Comment	Jan 21, 2026 1:40am

greptile-apps · 2026-01-20T22:03:48Z

Greptile Summary

This PR implements base64 string hydration for file outputs during workflow execution, enabling downstream blocks to access file content directly without additional fetches.

Key Changes:

Added base64 field to UserFile interface and exposed it through variable resolution
Created user-file-base64.server.ts with Redis caching (in-memory fallback) for base64 content
Files parsed via the API are now stored in execution storage with UserFile objects returned
All execution paths (API, streaming, scheduled, webhook, background) now support base64 hydration
Removed DNS pinning from external URL fetches in favor of using original URLs after DNS validation - this fixes TLS SNI issues for HTTPS connections while maintaining SSRF protection
Added proper log redaction for base64 content (truncated rather than fully logged)
Centralized isUserFile and related utilities into lib/core/utils/user-file.ts
Fixed createPinnedUrl to properly handle IPv6 addresses (brackets required in URLs)

Architecture:

Base64 hydration is opt-in via includeFileBase64 flag (defaults to true for most execution paths)
base64MaxBytes limits hydration to files under a size threshold (default 10MB)
Cache cleanup occurs at execution completion to prevent memory leaks

Confidence Score: 4/5

This PR is well-structured with proper error handling, caching, and cleanup mechanisms. The DNS pinning removal is a security improvement, not a regression.
The changes are comprehensive and touch many files, but follow consistent patterns. The new base64 hydration logic includes proper size limits, caching, and cleanup. The security changes (DNS pinning removal) are well-reasoned and maintain SSRF protection while fixing TLS issues.
The main files to verify during testing are user-file-base64.server.ts for caching behavior and streaming.ts for proper cleanup on all code paths.

Important Files Changed

Filename	Overview
apps/sim/lib/uploads/utils/user-file-base64.server.ts	New file implementing base64 hydration for UserFile objects with Redis/in-memory caching. Well-structured with proper error handling and cleanup.
apps/sim/app/api/files/parse/route.ts	Updated to store parsed files in execution storage and return UserFile objects. Includes proper execution context handling and error recovery.
apps/sim/app/api/workflows/[id]/execute/route.ts	Added includeFileBase64 and base64MaxBytes options to workflow execution. Properly cleans up cache after execution and hydrates outputs.
apps/sim/executor/variables/resolvers/block.ts	Enhanced output schema validation to properly handle file[] type arrays and validate accessible properties for UserFile objects.
apps/sim/lib/workflows/streaming/streaming.ts	Integrated base64 hydration into streaming responses with proper cache cleanup on completion and error paths.
apps/sim/lib/core/security/redaction.ts	Added TRUNCATED_MARKER for base64 fields in logs to prevent large data from appearing in logs while preserving other UserFile metadata.
apps/sim/lib/core/utils/user-file.ts	New utility file centralizing UserFile type checks and display field filtering. Includes base64 in exposed fields.
apps/sim/executor/execution/block-executor.ts	Added base64 hydration for block outputs when includeFileBase64 is enabled.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ExecuteRoute as /api/workflows/[id]/execute
    participant ExecutionCore
    participant BlockExecutor
    participant FileParseAPI as /api/files/parse
    participant Storage as Execution Storage
    participant Redis as Redis Cache

    Client->>ExecuteRoute: POST (includeFileBase64: true)
    ExecuteRoute->>ExecutionCore: executeWorkflowCore()
    ExecutionCore->>BlockExecutor: execute block (e.g., File Parser)
    BlockExecutor->>FileParseAPI: parse file
    FileParseAPI->>Storage: uploadExecutionFile()
    Storage-->>FileParseAPI: UserFile (with key)
    FileParseAPI-->>BlockExecutor: ParseResult with UserFile
    
    Note over BlockExecutor: if includeFileBase64 && containsUserFileWithMetadata
    BlockExecutor->>Redis: cache.get(file)
    alt Cache hit
        Redis-->>BlockExecutor: base64 string
    else Cache miss
        BlockExecutor->>Storage: downloadFileFromStorage()
        Storage-->>BlockExecutor: Buffer
        BlockExecutor->>BlockExecutor: bufferToBase64()
        BlockExecutor->>Redis: cache.set(file, base64, ttl)
    end
    BlockExecutor-->>ExecutionCore: output with UserFile.base64
    
    ExecutionCore-->>ExecuteRoute: ExecutionResult
    ExecuteRoute->>ExecuteRoute: hydrateUserFilesWithBase64(output)
    ExecuteRoute->>Redis: cleanupExecutionBase64Cache(executionId)
    ExecuteRoute-->>Client: Response with base64 hydrated files

icecrasher321 · 2026-01-20T23:45:38Z

@cursor review

icecrasher321 · 2026-01-21T00:21:42Z

@cursor review

icecrasher321 · 2026-01-21T00:34:33Z

@cursor review

icecrasher321 · 2026-01-21T01:38:38Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

* fix(google): wrap primitive tool responses for Gemini API compatibility (#2900) * fix(canonical): copilot path + update parent (#2901) * fix(rss): add top-level title, link, pubDate fields to RSS trigger output (#2902) * fix(rss): add top-level title, link, pubDate fields to RSS trigger output * fix(imap): add top-level fields to IMAP trigger output * improvement(browseruse): add profile id param (#2903) * improvement(browseruse): add profile id param * make request a stub since we have directExec * improvement(executor): upgraded abort controller to handle aborts for loops and parallels (#2880) * improvement(executor): upgraded abort controller to handle aborts for loops and parallels * comments * improvement(files): update execution for passing base64 strings (#2906) * progress * improvement(execution): update execution for passing base64 strings * fix types * cleanup comments * path security vuln * reject promise correctly * fix redirect case * remove proxy routes * fix tests * use ipaddr * feat(tools): added textract, added v2 for mistral, updated tag dropdown (#2904) * feat(tools): added textract * cleanup * ack pr comments * reorder * removed upload for textract async version * fix additional fields dropdown in editor, update parser to leave validation to be done on the server * added mistral v2, files v2, and finalized textract * updated the rest of the old file patterns, updated mistral outputs for v2 * updated tag dropdown to parse non-operation fields as well * updated extension finder * cleanup * added description for inputs to workflow * use helper for internal route check * fix tag dropdown merge conflict change * remove duplicate code --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> * fix(ui): change add inputs button to match output selector (#2907) * fix(canvas): removed invite to workspace from canvas popover (#2908) * fix(canvas): removed invite to workspace * removed unused props * fix(copilot): legacy tool display names (#2911) * fix(a2a): canonical merge (#2912) * fix canonical merge * fix empty array case * fix(change-detection): copilot diffs have extra field (#2913) * improvement(logs): improved logs ui bugs, added subflow disable UI (#2910) * improvement(logs): improved logs ui bugs, added subflow disable UI * added duplicate to action bar for subflows * feat(broadcast): email v0.5 (#2905) --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com>

icecrasher321 added 2 commits January 20, 2026 08:38

progress

05c4538

improvement(execution): update execution for passing base64 strings

c1200ef

fix types

0b67500

vercel bot temporarily deployed to Preview January 20, 2026 23:26 Inactive

cleanup comments

f151650

vercel bot temporarily deployed to Preview January 20, 2026 23:34 Inactive

This comment was marked as outdated.

Sign in to view

path security vuln

bcf65aa

vercel bot temporarily deployed to Preview January 21, 2026 00:20 Inactive

This comment was marked as outdated.

Sign in to view

reject promise correctly

7af5c18

vercel bot temporarily deployed to Preview January 21, 2026 00:34 Inactive

This comment was marked as outdated.

Sign in to view

icecrasher321 added 2 commits January 20, 2026 16:44

fix redirect case

257e049

remove proxy routes

5441ad9

vercel bot temporarily deployed to Preview January 21, 2026 01:19 Inactive

icecrasher321 added 2 commits January 20, 2026 17:28

fix tests

c85350f

use ipaddr

785e3a7

vercel bot deployed to Preview January 21, 2026 01:40 View deployment

cursor bot reviewed Jan 21, 2026

View reviewed changes

icecrasher321 merged commit 1f1f015 into staging Jan 21, 2026
11 checks passed

icecrasher321 deleted the feat/file-base64 branch January 21, 2026 01:49

waleedlatif1 mentioned this pull request Jan 21, 2026

v0.5.65: gemini updates, textract integration, ui updates #2909

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

improvement(files): update execution for passing base64 strings#2906

improvement(files): update execution for passing base64 strings#2906
icecrasher321 merged 10 commits intostagingfrom
feat/file-base64

icecrasher321 commented Jan 20, 2026

Uh oh!

vercel bot commented Jan 20, 2026 •

edited

Loading

Uh oh!

greptile-apps bot commented Jan 20, 2026

Uh oh!

icecrasher321 commented Jan 20, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

icecrasher321 commented Jan 20, 2026

Summary

Type of Change

Testing

Checklist

Uh oh!

vercel bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot commented Jan 20, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

icecrasher321 commented Jan 20, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

icecrasher321 commented Jan 21, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel bot commented Jan 20, 2026 •

edited

Loading